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[A] 


Image Split 


The optical device is to forms a pair of images in the camera 
with different points of view, thus forming a stereoscopic image 
by optical means. 


IBJ 


Image Capture Process 


The image capture process is performed by a standard low-cost 
camera with a standard USB of Firewire PC interface. The 
advantage of using a single camera to obtain stereo images is 
that the images are captured at the same time and with the same 
focal length of the lens, as well as the same spectral response, 
gain and most other parameters of the camera. The interfacing is 
simple and no synchronisation of more cameras is required. 


[C] 


Image Feature Extraction 
process 


The extraction process detects skin and colour of some objects 
attached to the person. (The process is detailed in Fig. 5) 


LDj 


3D Acquisition 


The Image Feature Extraction process is sub-divided. See Fig. 6 


LKJ 


3D Tracking 


The Image Feature Extraction process is sub-divided. See Fig. 7 




3D Model Fitting 


The observed and tracked 3D points are fitted to a 3D 
hierarchical model. 


[G] 


Model Parameter Extraction 


Parameters pertaining to the 3D model are extracted from the 
model 


LZJ 


Sensor System Calibration 


This process performs a calibration for the complete sensor 
system (both Camera and optical system). The calibration thus 
removes errors in the lens and mirrors. 



Fig. 4b 
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Captured image (from [B]) 
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Abstract 


[C.1] 


Camera Parameter Control 


The camera parameters are set (like gain, shutter-speed, image- 
size, gamma level, white balance) 


[C.2] 


Colour Detection 


The detection of skin follows a well known formula where the 
calculation is performed on each pixel. The calculation is a 
Boolean function of the value of the colours red, green and blue, 
RGB 


[C.3] 


Blob Processing 


Finding continuous areas of pixels of a given colour 


[C.4] 


Blob Validation 


A relation between blobs is established in order to test if the 
pairing is feasible further down the pipe-line. 


[C5] 


Blob Centre Coordinate Calc 


The centre of detected blobs are calculated 



Figo 5 
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Processed Images (from [C]) 
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[D.l] 


Correction Table Lookup and 
calculation 


For each blob-centre in a pair a lookup table entry determines 
the slope of the line of si^ht for both images of the camera. If 
the blob-centres correspond to an actual object then the position 
can be determined. Since the geometry of the camera and 
optical front-end is known, the calculation of the 3D position is 
done by tri angulation. 


[D.2] 


Formatting of 3D blob stream, 
including colours 


For every frame in the video-stream the calculated 3D-point set 
- together with their corresponding color-code - is formatted 
into a 'blob stream'. 



Fig„ 6 
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[E.1] 


Target Association 


Observed targets are associated with existing tracks 


[E.2] 


Target state estimation and 
prediction (Kalman) 


Trackstates are estimated and predicted by a linear 
Kalman filter 


[E.3] 


Track maintenance 


CRUD (Create, Read Update and Delete) operations are 
performed on tracks. 


[E.4] 


Labelling Heuristics 


Tracks are labelled according to some basic heuristics. 



Fig. 7 



